For deep learning applications a convolution kernel often operates on data that is sparse, meaning many of the values in the data equal zero. The sparsity can be either in the activations or in the weights. Sparsity in the activations results from rectified linear unit (ReLU) activation functions in a previous layer of the neural network. Sparsity in the weights occurs when the neural network has been pruned to increase accuracy or reduce the model size. Performing arithmetic operations on the elements having zero values is wasteful in terms of processing time and performance because the arithmetic operations do not contribute to the output. There is a need for addressing these issues and/or other issues associated with the prior art.